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\ / ABSTRACT  ./ 

In  this  paper,  we  preooivt-  a recursive  method^to  solve 


separable  differentiable  convex  knapsack  problems  with  bounded  vari- 
ables. The  method  differs  from  classical  optimization  algorithms  of 
convex  programming  and  determines  at  each  iteration  the  optimal  value 
of  at  least  one  variable.  Applications  of  such  problems  are  frequent 
in  resource  allocation  and  recently  have  shown  to  be  useful  in  hier- 


/ 


archical  production  planning.  Computational  results  are  presented. 
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ON  THE  SOLUTION  OF  CONVEX  KNAPSACK  PROBLEMS 
WITH  BOUNDED  VARIABLES 

Gabriel  R.  Bitran  and  Arnoldo  C.  Hax 
Massachusetts  Institute  of  Technology 

In  this  paper  we  present  a recursive  method  to  solve  the  problem 


(N)  : z = min  Z G.(x.) 

jej°  3 3 

Z x.  = Pc 

j ej°  3 


Hb  < x . < ub . 

j - J ~ J 


jeJ' 


(1) 

(2) 


x.  e X. 
J J 


jeJ0 


where  Gj(*)  j£J°  are  differentiable  convex  functions  on  the  open  convex  set 


C R jeJ°  respectively, 


ub,  > Jib,  j eJ°  and  Z 9b  < Pc 

i ~ i i£J°  j 

wise  the  problem  is  trivial  or  infeasible) . J 


< Z ub . (other- 
j ej°  3 


The  derivative  of  Gj  at  is  denoted  by  DG^(x^), 


The  iterative  method 
proposed  to  solve  (N)  differs  from  classical  convex  programming  algorithms 
and  at  each  iteration  determines  the  optimal  value  of  at  least  one  variable. 
This  problem  was  formerly  treated,  for  particular  objective  functions,  by 
convex  programming  arguments  [3]  and  dynamic  programming  [6].  More  recently 
Luss  and  Gupta  [4]  presented  an  iterative  method  for  strictly  convex  decreas- 
ing functions  and  a one  pass  algorithm  for  a set  of  particular  functions  with 
the  variables  bounded  from  below.  Luss  and  Gupta's  method  consists  in  relax- 
ing the  upper  bounding  constraints.  The  algorithm  proposed  in  this  paper 
applies  for  a more  general  set  of  functions.  At  each  iteration  it  solves  a 


simpler  problem  of  the  form  min  {Z  G^(x.)  st  Zx^ 


P,  x^eX^}  obtained  by 


relaxing  both  bounds  (2).  At  the  end  of  each  iteration  (except  possibly  the 
last)  we  show  that  either  the  subset  of  variables  (x 


j . x^  _>  ub ^ } have  optimal 
x^  J,bj } have  optimal  value  £b^  in  (N)  . 


value  ubj  in  (N)  or  the  subset  {x^ 

Applications  of  this  class  of  knapsack  problems  to  resource  allocation  can 
be  found  in  [3],  [4]  and  to  hierarchical  production  planning  in  [2], 

(N)  is  a convex  problem  and  is  regular  [1],  Thus  the  Kuhn-Tucker 
conditions  (3)-(9),  below,  are  necessary  and  sufficient  for  optimality  of 
x*  jej°  in  (N). 
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DGjt^)  t X + ^ - Tj  ■ 

0 

jeJ° 

(3) 

“j(VuV 

0 

jeJ° 

(4) 

T^tbj-X*) 

0 

jeJ° 

(5) 

E x*  = P° 
j£j°  J 

(6) 

tbJ  i xj  - ubj 

jeJ° 

(7) 

AeR,  u.  > 0,  Tj  £ 0 

j£J° 

(8) 

X3  £ Xj 

J EJ  ° 

(9) 

We  first  state  the  algorithm  and 

then 

prove  that  it  is  optimal. 

Algorithm 

Initialization:  J1  = J°,  P1  = Pc 


Iteration  3 (3=1, 2 ,3...) 

Step  1:  Solve  N(3)  : min  { £ X.CxJ  : I 


jeJ 


3 j j' 


jeJ 


3 j 


P3,  XjeX  jeJB} 


and  let  the  solution  be  xB  jeJB.  It  lb . < xB  < ub  IeJ^  define 

J J ~ j - j 

* s 3 * 

Xj  = Xj  jeJ  and  stop  the  solution  x^  jeJ  generated  by  the 

algorithm  is  optimal.  Otherwise  go  to  step  2. 


Step  2:  Compute 

A*  = 

and 
o3 


E o (xB-ub  ) where  JB  = (JeJB  : xB  > ub  } 
jej^  3 3 + 3 3 

l O (^b  -xB)  where  JB  = {jeJB  : xB  £ Jib  } 
jeJ_  33  “ 33 


If  AB  > define  x*  = ub^  jeJB  and  let 
A1  = /-A  PB+1-PB-Zftub  . 

+ jejJ  J 


Step  3: 
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h 
\ ' 


If  A^  < define  x.  = Jib  jeJ^  and  let 

j j " 


j^1  = jB_  jg,  p6+l 


pB"  S ft  «V 
Jed3  j 


3+1  * 

If  J = <)>  stop.  The  solution  jeJ°  generated  by  the  algorithm 
is  optimal.  Otherwise  let  3 = 3+1  and  go  to  step  1. 

Since  at  each  iteration  the  set  J is  reduced  by  at  least  one  element 
the  algorithm  is  finite.  To  prove  that  x^  generated  by  the  algorithm, 

solve  (N)  we  construct  a corresponding  Kuhn-Tucker  vector  through  the 
following  results.  Let  Ap  be  the  Kuhn-Tucker  multiplier  associated  to 
the  knapsack  contraint  in  N(3) . 

R R 

Lemma  1 : If  at  iteration  3 Ap  _>  Vp  then 

8 B 6 

a)  for  any  s£JP  we  have  -DG  (ubg)  _>  -DG^Cub^)  for  ail  ieJP-JP 

b)  »8+1  < »B 

Proof:  a)  A^  = -DGj(Xj)  jeJ^.  Let  seJ+  and  ieJ^-J^.  Then,  since 

for  convex  functions  [5] 


[ DG j ( Xj  > " DGj(xj)](x5  ” xj>  > 0 


(10) 


it  follows  that 


-DG  (ub  ) > -DG  (x6)  = A6 

S S — s s 


-DVV  > -DGi(ub1) 


b)  E x?  = p6-  ^ « X1  - pB"  Z 3 ubi  = pS+1  = E b+ixi+1 
j jej!  1 J jeJe+1  j 


e t3 


jeJ  -J 


oii  R+l  fi 

For  at  least  one  jejp  we  have  xP  — xj*  T^us 

A«+1  * ' 


where  the  inequality  follows  from  (10) . 


77J7K 


Lemma  2:  If  at  iteration  3 then 

a)  for  any  seJ^  we  have  -DG  (Jib  ) -DG  (Jib  ) for  all  ieJ^-J^ 

“ s s i i — 

b)  Ae+1  > A6 
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The  proofs  of  this  lemma  and  of  theorem  4 are  omitted  because  they 
similar  to  those  of  lemma  2 and  theorem  3 respectively. 


Theorem  3:  Assume  that  A^  >_  V8,  A*  < V3'  i=8+l , 8+2 , . . . , y-1 . Then 

a)  JY  D (J8-J8-J8) 


e>  AY 


b)  A 


Proof:  P 


8+1  - £ *s+1  - £ *?-  £ 6 “b,  + £ 6 *? 

1 i/'1  3 jEje-J®  JeJ+  J JEJ+  J 


,8+2 


- 6+2 

je/+2  Xj 


jeJ6+1  XJ+1  ' jLB+1  "“l 


1 xj  + 1 8+1  xi  + A Z 8+1  8/8  j 

jej8+1-/+I  jeJ-  iEj_ 


^ . 8+2  6+1  tB+1 

Thus,  since  J = J 


.B+2 


j£/+2  3 

Similarly  we  obtain 


x8  + A8  - Z ft+1  (£b  -xj). 

j£J8  3 j 


Y-1 


V 6 . .8 

1 X + A - ^ " g \~“4  “4 

jeJY  3 s=8+l  jejf  33 


(£b ,-x8) 


From  (11)  it  follows  that  for  at  least  one  joed 


6+1 


8+1 
*1 


8 

xj. 


Jo  o 

But,  from  the  Kuhn-Tucker  conditions  for  N(8+l) : 

,8+1 


/ 8+1 x ,8+1 

-DG  (x£  ) = h 


i£j 


Confining  (10),  (14),  and  (15)  it  follows  that 


8+1  ^ 6 f ,,  .,,.6+1 

Xj  _>  Xj  for  all  j£J 


are 


(ID 


(12) 


(13) 


(14) 


(15) 
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If  all  functions  are  not  strictly  convex  it  is  possible  that 

0+2_  B 

-DG,  (x,  ) = A = A . In  this  case  N(0+1)  may  have  more  than 

o o 

one  optimal  solution.  However  at  least  one  will  satisfy  this 
condition. 


Thus  J3+1  C J3  and  A3 


i «• 


(12) , (16)  and  the  fact  that  J3+2  = J3+^-J3+l  imply  that 
x3+2  > x3  for  all  jeJ3+2,  J3+2  c J3  and 

j - 1 - - - 

A3  - Z (4b. -x3)  - E (4b  -x3)  > 0 thus 

..,3+1  3 3 a . .3+2  J J 


v 3+3  _ 6 

lx.  > I x. 


Continuing  with  the  same  reasoning  we  obtain 


ji  C J3  i=3+l y-1;  A3  - Z Z (4b.-x3)>0 

s=0+l  j £JS  3 3 


and  from  (13) 


pY  * £ , ']  i S * 

jej'  J j £ J ' J 


g+2.  A ft 

These  conclusions  together  with  J = Jp  - JR  and 

,1+1  ,1  .1  J 0.1  1 , 

J = J -J_  i=0+l , . . . ,y-l  prove  part  a). 

From  (17)  it  follows  xY  ]>  x3  jeJY.  Thus,  from  (10)  and  (15)  with 
Y instead  of  0+1 


AY  = -DCj(xY)  < -DG  (xj)  = A3 


B B i i 

Theorem  A:  Assume  that  Ap  < VP,  A _>  V i=0+l,0+2, . . . ,Y~1.  Then 

a)  JY  2 (J3-J3-J3) 

b)  A3  < AY 
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Theorem  5:  The  set  Xj  jeJ°  generated  by  the  algorithm  is  optimal  in  (N) . 

* 

Proof:  By  lemmas  1 and  2,  theorems  3b)  and  4b)  the  set  x^  jeJ° 

generated  by  the  algorithm  has  the  following  property: 


-DG.  (ub,  )>...>  -DG,  (ub,  )>  -DG  (x*  )=...=  -DG  (x*  )>  -DG  (£b  )>...>  -DG  (£b.  ) 

k,  k — — k k — v,  v v — i,  i,  — — i is 

l 1 1 P P j { 1 1 g g,.  1 1 s 


~V" 


J 


corresponding  to  variables  corresponding  to  variables  corresponding  to  variables 
defined  at  upper  bound  not  defined  at  any  iteration  defined  at  lower  bound 

at  upper  or  lower  bound 


* 

! X 

j=l,2,...,g  are  obtained  at  the 

last 

iteration. 

Vj 

see 

that  conditions  ( 3) — ( 9 ) 

are  satisfied 

take 

X = -DG  ( 

•k 

\ ) 

V1 

vi 

Th 

= X + DG1  (£b  ) > 0 

3 1j 

0 

j=l,2, . . . ,s 

"'‘j 

= -DG^  (ub^  ) -X  _>  0 
j j 

* Tk  = 
j 

0 

J-1,2 

T 

\T 

= u =0  j=l ,2 , . 

Vj 

Vj 

Thus, since  x^  j£J°  also  satisfies  (6),  (7),  and  (9)  it  follows  that 
x^  = ub  j-1,2, . . . ,p;  x = x*  j=l,2 and 

j J i i 


x = £b  J*l,2,...,s  solve  (N) . 

J j 


MM 


The  algorithm  depends  strongly  on  the  existence  of  a solution  to 
N(0)  . However  when  among  the  G^(«)  jeJp  there  are  strictly  increasing 
and  decreasing  functions  N($)  has  no  solution.  The  next  theorem  shows 
how  to  cope  with  this  difficulty. 

Let  = {jeJ°:  G^(*)  is  strictly  increasing}  f 0 and 
J2  = G^-)  is  strictly  decreasing)  i 0 

Assume  that  (N)  has  an  optimal  solution  x^  j cf . Then 
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Theorem  6:  The  optimal  solution  of  (N)  is  such  that  either 

* * 
x^  = ub^  ie^  an<*'or  ~ ie^l“ 

Proof:  The  optimal  solution  satisfies  (3)-(9)  and  in  particular 

-DG^(ubj)  = A + Uj  jeJ(ub)  = {jeJ  : x^  *>  ub^} 

-DG  _,(«!>.)  = A - j.  jeJ(Jlb)  = { jeJ° : x*  = <.b  } and 


r r 


jeJ  - J (Jib)  - J (ub)  . 


Note  that  the  assumption  P°  > E Lb  implies  J°-J(Lb)  £ 0. 

* J£J°  j 

If  min  {-DGj(Xj)  j£j°-J(Lb)}  >_  0,  since  -DG^Cx^)  < 0 iEj^ , 


it  follows  that  C J(Lb).  Otherwise,  since  -DG^(x^)  > 0 iEJ2 

and  -DG . (ub  ) .>  -DG.(x*)  _>  -DG  («.b  ) for  any  jeJ(ub)  iEJ°-J(ub)-J(f.b) 
J J * * k.  K. 

and  k£J(£b)  , we  have  that  ^ C J (ub) 


A direct  consequence  of  theorem  6 is  that  the  solution  to  (N)  can 
be  obtained  by  solving  the  following  two  problems 


and 


(N.  ) : z.  = min{  £ G.(x,):  £ x.  = P°-  £ ub . £b  <x._<ub.  j£j°-J  } 

jeJ°-J2  3 3 j£J°-J2  3 j£J2  3 3 3 j 2 


(N~)  : z0  = min{  £ G.(x. ):  £ x.  = P°-  £ Lb.  Lb  <x  <ub . j£j°-J, } 

2 2 3 j 3 j£J1  3 3~  3~  3 1 


and  taking  z = mintz^.z,,). 

Let  (N<)  and  (N>)  be  the  versions  of  (N)  when  constraint  (1)  is  an 
inequality  of  the  type  and  respectively.  For  a variable  jeJ0  let  h^ 
be  the  value  of  x.  for  which  DG.(x.)  = 0.  If  such  a point  does  not  exist 

3 3 3 o 

we  adopt  h.  = -“>  (+°°)  in  Theorem  7 (Theorem  8).  Let  x.  JeJ  solve  (N) . 
Then 


Theorem  7 : a)  If  A = -DG  (x  ) 0,  x.  = x j£j°  solve  (N>) . 

j Vj  33 


b)  If  A = -DG  (x  ) < 0,  x, 

J j 3 


defined  as: 
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xi  = £bi  J=l»2.*..,s; 

j j 

x = max(£b  ,h  ) j=l,2,...,g; 

j j j 

x,  = max(£b  ,h  ) for  all  k.  such  that  DG,  (ub  ) 

i ki  j j kj  kj 

x,  = ub  for  all  k.  such  that  DT,  (ub,  ) < 0 

j J kj  kj 

solve  (N<)  . 


Theorem  8:  a) 

If  A = 

-DG 

Vj 

(xv  ) £ 0,  x,  = ■<.  j£J°  solves  (N>)  . 

j 3 

b) 

If  A = 

-DG 

* 

(x  ) > 0,  x.  defined  as: 

Vj 

< 

<_u 

(_». 

V = 

Ubk 

j 1,2, ... tp; 

j 

j 

x = min(ub  ,h  ) j=l,2,...,g; 


x = min(ub  ,h  ) for  all  i such  that  DG  ( £b  ) 

i i 1 j 

x^  = Jib  for  all  i such  that  DG  (Jib.  ) > 0 

j j 3 j ii 


> 0 and 


< 0 and 


solve  (N>) . 
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Computational  Results 

In  Tables  1 to  7 we  present  the  results  of  84  problems  of  type  (N) . 
The  data  were  randomly  generated.  For  Identification  purposes,  our 
algorithm  was  denominated  Bitran-Hax.  In  each  problem  the  objective 
function  was  composed  by  functions  G^(x^)  of  the  same  family.  They  are 
indicated  in  the  first  row  of  each  table.  Following  the  time  in  seconds 
to  solve  each  problem,  by  both  methods,  appears  the  number  of  iterations 
required,  n represents  the  number  of  variables  in  a problem.  For  a fixed 
n three  problems  were  solved  for  each  type  of  objective  function.  In 
Luss  and  Gupta's  method  [4]  the  ordering  of  the  derivatives  evaluated  at 
the  lower  bound  of  each  variable  was  executed  by  the  "Quick  Method". 

In  our  algorithm,  problems  N(S)  were  solved  through  the  corresponding 
Kuhn-Tucker  conditions.  For  the  problems  of  Tables  6 and  7,  Luss  and 
Gupta's  algorithm  does  not  apply  because  in  the  former  one  the  objective 
functions  are  strictly  convex  increasing  and  in  the  later  one  we  have 
not  imposed  any  condition  among  the  values  of  the  bounds,  P°  and  the 
point  where  each  of  the  G^(x^)  attains  its  minimum.  The  computer  used 
is  a Borroughs  B6700.  The  programs  were  written  in  Algol.  Application 
of  problems  presented  in  Tables  5 and  7 to  hierarchical  production 
planning  can  be  found  in  [2].  The  parameters  (s^ .m^ , £b^ ,ub^ ,etc . . . . ) 
corresponding  to  problems  in  Tables  1 to  5 (6  and  7)  were  randomly 
generated  in  intervals  where  the  functions  G^(x^)  are  strictly  convex 
decreasing  (strictly  convex) . 


exp 
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